Discovering Dialog Rules by means of an Evolutionary Approach

Designing the rules for the dialog management process is oneof the most resources-consuming tasks when developing a dialog system. Although statistical approaches to dialog management are becoming mainstream in research and industrial contexts, still many systems are being developed following the rule-based or hybrid paradigms. For example, when developers require deterministic system responses to keep total control on the decisions made by the system, or because the infrastructure employed is designed for rule-based systems using technologies currently used in commercial platforms. In this paper, we propose the use of evolutionary algorithms to automatically obtain the dialog rules that are implicit in a dialog corpus. Our proposal makes it possible to exploit the benefits of statistical approaches to build rule-based systems. Our proposal has been evaluated with a practical spoken dialog system, for which we have automatically obtained a set of fuzzy rules to successfully manage the dialog.The research leading to these results has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 823907 (MENHIR project:https://menhir-project.eu

An approach to develop intelligent learning environments by means of immersive virtual worlds

Merging Immersive Virtual Environments, Natural Language Processing and Artificial Intelligence techniques provides a number of advantages to develop Intelligent Environments for multiple applications. This paper is focused on the application of these technologies to develop intelligent learning environments. Education is one of the most interesting applications of immersive virtual environments, as their flexibility can be exploited in order to create heterogeneous groups from all over the world who can collaborate synchronously in different virtual spaces. We highlight the potential of virtual worlds as an educative tool and propose a model to create learning environments within Second Life or OpenSimulator combining the Moodle learning management system, embodied conversational metabots, and programmable 3D objects. Our proposal has been applied in several subjects of the Computer Science degree in the Carlos III University of Madrid. The results of the evaluation show that developed learning environment fosters engagement and collaboration and helps students to better understand complex concepts.Spanish Government TEC2012-37832-C02-01Consejo Interinstitucional de Ciencia y Tecnologia (CICYT) TEC2011-28626-C02-02Project CAM CONTEXTS S2009/TIC-148

A Neural Network Approach to Intention Modeling forUser-Adapted Conversational Agents

Spoken dialogue systems have been proposed to enable a more natural and intuitive interaction with the environment andhuman-computer interfaces. In this contribution, we present a framework based on neural networks that allows modeling of theuser’s intention during the dialogue and uses this prediction todynamically adapt the dialoguemodel of the system taking intoconsideration the user’s needs and preferences. We have evaluated our proposal to develop a user-adapted spoken dialogue systemthat facilitates tourist information and services and provide a detailed discussion of the positive influence of our proposal in thesuccess of the interaction, the information and services provided, and the quality perceived by the users

An approach to develop intelligent learning environments by means of immersive virtual worlds

Merging Immersive Virtual Environments, Natural Language Processing and Artificial Intelligence techniques provides a number of advantages to develop Intelligent Environments for multiple applications. This paper is focused on the application of these technologies to develop intelligent learning environments. Education is one of the most interesting applications of immersive virtual environments, as their flexibility can be exploited in order to create heterogeneous groups from all over the world who can collaborate synchronously in different virtual spaces. We highlight the potential of virtual worlds as an educative tool and propose a model to create learning environments within Second Life or OpenSimulator combining the Moodle learning management system, embodied conversational metabots, and programmable 3D objects. Our proposal has been applied in several subjects of the Computer Science degree in the Carlos III University of Madrid. The results of the evaluation show that developed learning environment fosters engagement and collaboration and helps students to better understand complex concepts.This work was supported in part by Projects MINECO TEC2012-37832-C02-01, CICYT TEC2011-28626-C02-02, CAM CONTEXTS (S2009/TIC-1485).Publicad

Bringing together commercial and academic perspectives for the development of intelligent AmI interfaces

The users of Ambient Intelligence systems expect an intelligent behavior from their environment, receiving adapted and easily accessible services and functionality. This can only be possible if the communication between the user and the system is carried out through an interface that is simple (i.e. which does not have a steep learning curve), fluid (i.e. the communication takes place rapidly and effectively), and robust (i.e. the system understands the user correctly). Natural language interfaces such as dialog systems combine the previous three requisites, as they are based on a spoken conversation between the user and the system that resembles human communication. The current industrial development of commercial dialog systems deploys robust interfaces in strictly defined application domains. However, commercial systems have not yet adopted the new perspective proposed in the academic settings, which would allow straightforward adaptation of these interfaces to various application domains. This would be highly beneficial for their use in AmI settings as the same interface could be used in varying environments. In this paper, we propose a new approach to bridge the gap between the academic and industrial perspectives in order to develop dialog systems using an academic paradigm while employing the industrial standards, which makes it possible to obtain new generation interfaces without the need for changing the already existing commercial infrastructures. Our proposal has been evaluated with the successful development of a real dialog system that follows our proposed approach to manage dialog and generates code compliant with the industry-wide standard VoiceXML.Research funded by projects CICYT TIN2011-28620-C02-01, CICYT TEC2011-28626-C02-02, CAM CONTEXTS (S2009/TIC-1485), and DPS2008- 07029-C02-02.Publicad

Providing personalized Internet services by means of context-aware spoken dialogue systems

The widespread use of new mobile technology implementing wireless communications enables a new type of advanced applications to access information services on the Internet. In order to provide services which meet the user needs through intelligent information retrieval, the system must sense and interpret the user environment and the communication context. Though context-awareness is vital to provide services adapted to the user preferences, it cannot be useful if such services are difficult to access. The development of spoken dialogue systems for these applications facilitates interaction in natural language with the environment which is also benefited from contextual information. In this paper, we propose a framework to develop context-aware dialogue systems that dynamically incorporate user specific requirements and preferences as well as characteristics about the interaction environment, in order to improve and personalize web information and services. We have identified the major components for context-aware dialogue systems and placed them within a general-purpose architecture. The framework also describes a representation mode based on a dialogue register in order to share information between the elements of the architecture, and incorporates statistical methodologies for dialogue management in order to reduce the effort required for both the implementation of a new system and the adaptation to a new task. We have evaluated our proposal developing a travel-planning system, and provide a detailed discussion of its positive influence in the quality of the interaction and the information and services provided.Research funded by projects CICYT TIN2011- 28620-C02-01, CICYT TEC2011-28626-C02-02, CAM CONTEXTS (S2009/TIC-1485), and DPS2008- 07029-C02-02.Publicad

The role of trust in proactive conversational assistants

Humans and machines harmoniously collaborating and bene ting from each other is a long lasting dream for researchers in robotics and arti cial intelligence. An important feature of ef cient and rewarding cooperation is the ability to assume possible problematic situations and act in advance to prevent negative outcomes. This concept of assistance is known under the term proactivity. In this article, we investigate the development and implementation of proactive dialogues for fostering a trustworthy human-computer relationship and providing adequate and timely assistance. Here, we make several contributions. A formalisation of proactive dialogue in conversational assistants is provided. The formalisation forms a framework for integrating proactive dialogue in conversational applications. Additionally, we present a study showing the relations between proactive dialogue actions and several aspects of the perceived trustworthiness of a system as well as effects on the user experience. The results of the experiments provide signi cant contributions to the line of proactive dialogue research. Particularly, we provide insights on the effects of proactive dialogue on the human-computer trust relationship and dependencies between proactive dialogue and user specific and situational characteristics

Incorporating android conversational agents in m-learning apps

Smart Mobile Devices Have Fostered New Learning Scenarios That Demand Sophisticated Interfaces. Multimodal Conversational Agents Have Became A Strong Alternative To Develop Human-Machine Interfaces That Provide A More Engaging And Human-Like Relationship Between Students And The System. The Main Developers Of Operating Systems For Such Devices Have Provided Application Programming Interfaces For Developers To Implement Their Own Applications, Including Different Solutions For Developing Graphical Interfaces, Sensor Control And Voice Interaction. Despite The Usefulness Of Such Resources, There Are No Strategies Defined For Coupling The Multimodal Interface With The Possibilities That These Devices Offer To Enhance Mobile Educative Apps With Intelligent Communicative Capabilities And Adaptation To The User Needs. In This Paper, We Present A Practical M-Learning Application That Integrates Features Of Android Application Programming Interfaces On A Modular Architecture That Emphasizes Interaction Management And Context-Awareness To Foster User-Adaptively, Robustness And Maintainability.This work was supported in part by Projects MINECO TEC2012-37832-C02-01, CICYT TEC2011-28626-C02-02, CAM CONTEXTS (S2009/TIC-1485

Utilización de los sistemas de diálogo hablado para el acceso a la información en diferentes dominios

Ponencias de la Segunda Conferencia internacional sobre brecha digital e inclusión social, celebrada del 28 al 30 de octubre de 2009 en la Universidad Carlos III de MadridLa acción de conversar es el modo más natural para resolver un gran número de acciones cotidianas entre los seres humanos. Por este motivo, un interés histórico dentro del campo de las Tecnologías del Habla ha sido utilizar estas tecnologías en aplicaciones reales, especialmente en aplicaciones que permitan a una persona utilizar su voz para obtener información mediante la interacción directa con una máquina o para controlar un determinado sistema. El objetivo es disponer de sistemas que faciliten la comunicación persona-máquina del modo más natural posible, es decir, a través de la conversación. En esta comunicación se resumen los resultados de la aplicación de estas tecnologías para el desarrollo de diferentes sistemas de diálogo en los que la interacción entre el usuario y el sistema se lleva a cabo mediante habla espontánea en castellano. Para su implementación se ha primado la utilización de diferentes herramientas de software libre para el reconocimiento automático del habla, compresión del lenguaje natural, gestión del diálogo y síntesis de texto a voz. De este modo, el objetivo principal de la comunicación es presentar las principales ventajas que proporcionan los sistemas de diálogo para facilitar el acceso a diferentes servicios dentro de dominios semánticos restringidos, qué posibilidades brinda el uso de herramientas de software libre para su implementación y su evaluación en diferentes casos concretos de aplicación

A Proposal for Multimodal Emotion Recognition Using Aural Transformers and Action Units on RAVDESS Dataset

The work leading to these results was supported by the Spanish Ministry of Science and Innovation through the projects GOMINOLA (PID2020-118112RB-C21 and PID2020-118112RB-C22, funded by MCIN/AEI/10.13039/501100011033), CAVIAR (TEC2017-84593-C2-1-R, funded by MCIN/AEI/10.13039/501100011033/FEDER "Una manera de hacer Europa"), and AMIC-PoC (PDC2021-120846-C42, funded by MCIN/AEI/10.13039/501100011033 and by "the European Union "NextGenerationEU/PRTR"). This research also received funding from the European Union's Horizon2020 research and innovation program under grant agreement No 823907 (http://menhir-project.eu, accessed on 17 November 2021). Furthermore, R.K.'s research was supported by the Spanish Ministry of Education (FPI grant PRE2018-083225).Emotion recognition is attracting the attention of the research community due to its multiple applications in different fields, such as medicine or autonomous driving. In this paper, we proposed an automatic emotion recognizer system that consisted of a speech emotion recognizer (SER) and a facial emotion recognizer (FER). For the SER, we evaluated a pre-trained xlsr-Wav2Vec2.0 transformer using two transfer-learning techniques: embedding extraction and fine-tuning. The best accuracy results were achieved when we fine-tuned the whole model by appending a multilayer perceptron on top of it, confirming that the training was more robust when it did not start from scratch and the previous knowledge of the network was similar to the task to adapt. Regarding the facial emotion recognizer, we extracted the Action Units of the videos and compared the performance between employing static models against sequential models. Results showed that sequential models beat static models by a narrow difference. Error analysis reported that the visual systems could improve with a detector of high-emotional load frames, which opened a new line of research to discover new ways to learn from videos. Finally, combining these two modalities with a late fusion strategy, we achieved 86.70% accuracy on the RAVDESS dataset on a subject-wise 5-CV evaluation, classifying eight emotions. Results demonstrated that these modalities carried relevant information to detect users’ emotional state and their combination allowed to improve the final system performance.Spanish Government PID2020-118112RB-C21 PID2020-118112RB-C22 MCIN/AEI/10.13039/501100011033 TEC2017-84593-C2-1-R MCIN/AEI/10.13039/501100011033/FEDER PDC2021-120846-C42European Union "NextGenerationEU/PRTR")European Union's Horizon2020 research and innovation program 823907German Research Foundation (DFG) PRE2018-08322